# Small-scale efficient training

orca_mini_3b is a text generation model trained on the OpenLLaMa-3B model, utilizing instructions and inputs from the WizardLM, Alpaca, and Dolly-V2 datasets for explanatory fine-tuning, and applying the dataset construction methods from the Orca research paper.

Kaz-RoBERTa is a transformers model pre-trained in a self-supervised manner on a large-scale Kazakh corpus, primarily designed for masked language modeling tasks.